{
    "cells": [
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<a href=\"https://cocl.us/DL0320EN_TOP_IMAGE\">\n    <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0320EN/Assets/Images/Top.png\" width=\"750\" alt=\"IBM 10TB Storage\" />\n</a>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h1>Fashion-MNIST Project </h1>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2>Table of Contents</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<p>In this project, you will classify  Fashion-MNIST dataset using convolutional neural networks.</p>\n<ul>\n  \n<ul>\n<li><a href=\"#Preparation\">Preparation</a></li>\n<li><a href=\"#Q1\">Questions 1: Create a Dataset Class</li>\n<li><a href=\"#Train\">Define Softmax, Criterion function, Optimizer and Train the Model</a></li>\n\n</ul>\n \n\n</ul>\n\n<p>Estimated Time Needed: <b>30 min</b></p>\n<hr>"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "vml_credentials = {\n  \"apikey\": \"b3JYVhVZ_CntKJsOIISzNInjwmPWYdqYIKqpYZuvkP6E\",\n  \"iam_apikey_description\": \"Auto-generated for key 6411099f-9359-4665-a82f-d09c1920e669\",\n  \"iam_apikey_name\": \"Service credentials-1\",\n  \"iam_role_crn\": \"crn:v1:bluemix:public:iam::::serviceRole:Writer\",\n  \"iam_serviceid_crn\": \"crn:v1:bluemix:public:iam-identity::a/a1f545ade2eb4bc5821e1b54fa1be801::serviceid:ServiceId-54dff849-23b8-4c42-b8cf-bfd9cf40219d\",\n  \"instance_id\": \"a71b40e2-b4cb-4faf-8cea-aaf93cb5e3c1\",\n  \"url\": \"https://us-south.ml.cloud.ibm.com\"\n}",
            "execution_count": 1,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "cos_credentials = {\n  \"apikey\": \"0Slh2ipCqdj_TTaj1-YjpoOavKzzTZApQPwdA47RkAfg\",\n  \"cos_hmac_keys\": {\n    \"access_key_id\": \"04bde41a442f4d05b109a537cf550db2\",\n    \"secret_access_key\": \"311aaf5bb38424d84f012af01020feb01ade8b7287603a14\"\n  },\n  \"endpoints\": \"https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints\",\n  \"iam_apikey_description\": \"Auto-generated for key 04bde41a-442f-4d05-b109-a537cf550db2\",\n  \"iam_apikey_name\": \"pytorch\",\n  \"iam_role_crn\": \"crn:v1:bluemix:public:iam::::serviceRole:Writer\",\n  \"iam_serviceid_crn\": \"crn:v1:bluemix:public:iam-identity::a/a1f545ade2eb4bc5821e1b54fa1be801::serviceid:ServiceId-30f67cf9-9898-4984-9dc6-c2343d873b61\",\n  \"resource_instance_id\": \"crn:v1:bluemix:public:cloud-object-storage:global:a/a1f545ade2eb4bc5821e1b54fa1be801:f64de2a3-66b6-4123-821f-353904cefb45::\"\n}",
            "execution_count": 2,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "\n\n#Install PyTorch\n!pip install http://download.pytorch.org/whl/cpu/torch-0.4.1-cp35-cp35m-linux_x86_64.whl\n#Install torchvision\n!pip install torchvision\n\n",
            "execution_count": 4,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "\u001b[31mERROR: torch-0.4.1-cp35-cp35m-linux_x86_64.whl is not a supported wheel on this platform.\u001b[0m\nCollecting torchvision\n\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7e/90/6141bf41f5655c78e24f40f710fdd4f8a8aff6c8b7c6f0328240f649bdbe/torchvision-0.5.0-cp36-cp36m-manylinux1_x86_64.whl (4.0MB)\n\u001b[K     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 4.0MB 19.0MB/s eta 0:00:01\n\u001b[?25hCollecting torch==1.4.0 (from torchvision)\n\u001b[?25l  Downloading https://files.pythonhosted.org/packages/24/19/4804aea17cd136f1705a5e98a00618cb8f6ccc375ad8bfa437408e09d058/torch-1.4.0-cp36-cp36m-manylinux1_x86_64.whl (753.4MB)\n\u001b[K     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588| 753.4MB 41kB/s s eta 0:00:01MB/s eta 0:00:14     |\u2588\u2588\u2588\u2588\u2588\u2588\u258d                         | 151.0MB 20.7MB/s eta 0:00:30     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258b                 | 345.2MB 16.8MB/s eta 0:00:25     |\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u2588\u258a            | 464.7MB 53.6MB/s eta 0:00:06\n\u001b[?25hRequirement already satisfied: pillow>=4.1.1 in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (5.4.1)\nRequirement already satisfied: six in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (1.12.0)\nRequirement already satisfied: numpy in /opt/conda/envs/Python36/lib/python3.6/site-packages (from torchvision) (1.15.4)\nInstalling collected packages: torch, torchvision\nSuccessfully installed torch-1.4.0 torchvision-0.5.0\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"Preparation\" >Preparation</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Download the datasets you needed for this lab."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "The following are the PyTorch modules you are going to need"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# PyTorch Modules you need for this lab\n\nfrom torch.utils.data import Dataset, DataLoader\n\nfrom torchvision import transforms\nimport torch \nimport torch.nn as nn\nimport torchvision.transforms as transforms\nimport torchvision.datasets as dsets\ntorch.manual_seed(0)",
            "execution_count": 5,
            "outputs": [
                {
                    "output_type": "execute_result",
                    "execution_count": 5,
                    "data": {
                        "text/plain": "<torch._C.Generator at 0x7f5167976ed0>"
                    },
                    "metadata": {}
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Import Non-PyTorch Modules "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "# Other non-PyTorch Modules\n\nfrom matplotlib.pyplot import imshow\nimport matplotlib.pylab as plt\n\nfrom PIL import Image",
            "execution_count": 6,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "def show_data(data_sample):\n    plt.imshow(data_sample[0].numpy().reshape(IMAGE_SIZE, IMAGE_SIZE), cmap='gray')\n    plt.title('y = '+ str(data_sample[1].item()))",
            "execution_count": 7,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<hr>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<hr>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"Questions 1\">Questions 1: Create a Dataset Class</h2>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "In this section, you will load a Dataset object, but first you must transform the dataset. Use the <code>Compose</code> function perform the following transforms:. \n<ol>\n    <li>use the transforms object to<code> Resize </code> to resize the image.</li>\n    <li>use the transforms object to<code> ToTensor </code> to concert the image to a tensor.</li>\n</ol>\n\nYou will then take a screen shot of your validation data."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Use the compose function ot compse the "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "#Hint:\n\nIMAGE_SIZE = 16\n\ntransforms.Resize((IMAGE_SIZE, IMAGE_SIZE)),\ntransforms.ToTensor()#\ncomposed = transforms.Compose([transforms.Resize((IMAGE_SIZE, IMAGE_SIZE)), transforms.ToTensor()])",
            "execution_count": 8,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<hr>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Create two dataset objects for the Fashion MNIST  dataset. One for training data called <code> dataset_train </code> and one for validation data <code>dataset_val</code>. You will be asked to take a screenshot of several samples."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<b>Hint:</b>\n<code>dsets.FashionMNIST(root= './data', train=???, transform=composed,  download=True)</code>"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "\ndataset_train=dsets.FashionMNIST(root= '.fashion/data', train=True, transform=composed,  download=True)\ndataset_val=dsets.FashionMNIST(root= '.fashion/data', train=False, transform=composed, download=True)\n",
            "execution_count": 9,
            "outputs": [
                {
                    "output_type": "stream",
                    "text": "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to .fashion/data/FashionMNIST/raw/train-images-idx3-ubyte.gz\n",
                    "name": "stdout"
                },
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))",
                        "application/vnd.jupyter.widget-view+json": {
                            "version_major": 2,
                            "version_minor": 0,
                            "model_id": "12d6b0a5ddf44dec9fc580d49efad21b"
                        }
                    },
                    "metadata": {}
                },
                {
                    "output_type": "stream",
                    "text": "\nExtracting .fashion/data/FashionMNIST/raw/train-images-idx3-ubyte.gz to .fashion/data/FashionMNIST/raw\nDownloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to .fashion/data/FashionMNIST/raw/train-labels-idx1-ubyte.gz\n",
                    "name": "stdout"
                },
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))",
                        "application/vnd.jupyter.widget-view+json": {
                            "version_major": 2,
                            "version_minor": 0,
                            "model_id": "60fa24c545964e90ac5681e076d5c41b"
                        }
                    },
                    "metadata": {}
                },
                {
                    "output_type": "stream",
                    "text": "\nExtracting .fashion/data/FashionMNIST/raw/train-labels-idx1-ubyte.gz to .fashion/data/FashionMNIST/raw\nDownloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to .fashion/data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz\n",
                    "name": "stdout"
                },
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))",
                        "application/vnd.jupyter.widget-view+json": {
                            "version_major": 2,
                            "version_minor": 0,
                            "model_id": "df42df3facad45d59436e80a39c3a63f"
                        }
                    },
                    "metadata": {}
                },
                {
                    "output_type": "stream",
                    "text": "\nExtracting .fashion/data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to .fashion/data/FashionMNIST/raw\nDownloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to .fashion/data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz\n",
                    "name": "stdout"
                },
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "HBox(children=(IntProgress(value=1, bar_style='info', max=1), HTML(value='')))",
                        "application/vnd.jupyter.widget-view+json": {
                            "version_major": 2,
                            "version_minor": 0,
                            "model_id": "69de4cc7c5684ae6ab9ac5722836949a"
                        }
                    },
                    "metadata": {}
                },
                {
                    "output_type": "stream",
                    "text": "\nExtracting .fashion/data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to .fashion/data/FashionMNIST/raw\nProcessing...\nDone!\n",
                    "name": "stdout"
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "for n,data_sample in enumerate(dataset_val):\n\n    show_data(data_sample)\n    plt.show()\n    if n==2:\n        break ",
            "execution_count": 10,
            "outputs": [
                {
                    "output_type": "error",
                    "ename": "AttributeError",
                    "evalue": "'int' object has no attribute 'item'",
                    "traceback": [
                        "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
                        "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
                        "\u001b[0;32m<ipython-input-10-f21453ba8209>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mdata_sample\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdataset_val\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m     \u001b[0mshow_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata_sample\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m     \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mn\u001b[0m\u001b[0;34m==\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
                        "\u001b[0;32m<ipython-input-7-72abd8659c69>\u001b[0m in \u001b[0;36mshow_data\u001b[0;34m(data_sample)\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mshow_data\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata_sample\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m     \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimshow\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata_sample\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnumpy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mIMAGE_SIZE\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mIMAGE_SIZE\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcmap\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'gray'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m     \u001b[0mplt\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtitle\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'y = '\u001b[0m\u001b[0;34m+\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata_sample\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
                        "\u001b[0;31mAttributeError\u001b[0m: 'int' object has no attribute 'item'"
                    ]
                },
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "<Figure size 432x288 with 1 Axes>",
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAADzRJREFUeJzt3XuMXOV9xvHvw9oGX7C9tjFxMBeDDBLXApZFTESjulCHWjaV8odRo7ohkhW1tFApSpwiNVGlSk3T+0WJXEJKWwRRCTQogsYWSdpK1C6La26xg42hYDCQlNqOHWFj8+sfc5yOl5n1nPdcvJv3+Uirub3vvL89s8+eM2fmnFcRgZnl57RTXYCZnRoOv1mmHH6zTDn8Zply+M0y5fCbZcrhN8uUw2+WKYffLFOT2hxMkr9OaNawiNAg7bzmN8uUw2+WqUrhl7RC0g8k7ZK0vq6izKx5Sj2qT9IQ8AJwI7AHeBK4NSK+P0Yfv+c3a1gb7/mXArsiYndEHAEeAFZXeD4za1GV8J8DvNp1e09xn5lNAFU+6uu1afG+zXpJ64B1FcYxswZUCf8e4Nyu2wuB10c3iogNwAbwe36z8aTKZv+TwGJJiyRNAdYAj9RTlpk1LXnNHxFHJd0OfBsYAu6JiOdrq8zMGpX8UV/SYN7sN2ucv95rZmNy+M0y5fCbZcrhN8uUw2+WKYffLFMOv1mmHH6zTDn8Zply+M0y5fCbZcrhN8uUw2+WKYffLFMOv1mmHH6zTDn8Zply+M0y5fCbZSo5/JLOlfRdSdslPS/pjjoLM7NmVZmrbwGwICK2SjoTeAq4xXP1mZ1ajZ/AMyL2RsTW4vqPge14ui6zCaPKjD0/JekC4GpgS4/HPF2X2ThU+bz9kmYA/wr8QUQ8dJK23uw3a1gr5+2XNBn4BnDfyYJvZuNLlR1+Au4F3o6IOwfs4zW/WcMGXfNXCf+HgX8HngXeK+7+3Yh4dIw+Dr9ZwxoPfwqH36x5nqvPzMZUy0d9Zrno7Opqp99777138kYVeM1vlimH3yxTDr9Zphx+s0w5/GaZcvjNMuXwm2XK4TfLlMNvlimH3yxTDr9Zphx+s0z5wB4bU+qBLHPnzi3d59ixY0lj7du3r3Sf1EPZTzstbX25ePHi0n127txZuk+ZZeg1v1mmHH6zTDn8ZpmqHH5JQ5L+S9K36ijIzNpRx5r/Djqz9ZjZBFL1vP0LgV8G7q6nHDNrS9U1/58Dn+H/T91tZhNElSm6VwJvRcRTJ2m3TtKIpJHUscysflXW/NcDqyS9DDwA/IKkfxzdKCI2RMSSiFhSYSwzq1mVKbo/FxELI+ICYA3wnYj4eG2VmVmj/Dm/WaZq+W5/RHwP+F4dz2Vm7fCa3yxTPqovE6lH502bNi2p38qVK0v3mT17dtJYmzdvLt3nyJEjSWPNnz8/qd95551Xus/u3btL9ykzxZfX/GaZcvjNMuXwm2XK4TfLlMNvlimH3yxTDr9Zphx+s0w5/GaZcvjNMuXwm2XK4TfLlMNvlikf1ZeJSZPSXuply5Yl9bviiitK97nooouSxlq1alVSvxSHDh1K6rdjx47SfYaGhkr3effddwdu6zW/WaYcfrNMVZ20Y7akByXtkLRd0ofqKszMmlX1Pf9fAP8SER+TNAVIO+2LmbUuOfySZgI3AL8OEBFHgLRzI5lZ66ps9l8I/BD4WjFL792SptdUl5k1rEr4JwHXAF+OiKuBQ8D60Y08XZfZ+FQl/HuAPRGxpbj9IJ1/BifwdF1m41OV6breAF6VdElx13Lg+7VUZWaNq7q3/7eA+4o9/buBT1QvyczaUCn8EbEN8Oa82QTkb/iZZcoH9tQgdSqsVCkHfFx11VVJYy1dujSp3zvvvFO6T5mpprrt37+/dJ/h4eGksSIiqd+8efNa6fPGG28M3NZrfrNMOfxmmXL4zTLl8JtlyuE3y5TDb5Yph98sUw6/WaYcfrNMOfxmmXL4zTLl8JtlyuE3y9SEOKqv7aPmykqtb8qUKUn9rrzyytJ9li9fnjTWGWeckdRv5syZpfscOZJ28ueU5Xj48OGksY4ePZrUL+UIvZQjMQ8cODBwW6/5zTLl8Jtlqup0Xb8j6XlJz0m6X1LaNqKZtS45/JLOAX4bWBIRlwNDwJq6CjOzZlXd7J8ETJU0ic48fa9XL8nM2lDlvP2vAX8MvALsBfZHxMa6CjOzZlXZ7B8GVgOLgA8C0yV9vEc7T9dlNg5V2ez/ReCliPhhRLwLPAQsG93I03WZjU9Vwv8KcJ2kaep8y2U5sL2essysaVXe82+hMznnVuDZ4rk21FSXmTWs6nRdnwc+X1MtZtYif8PPLFMOv1mmWj+qL+UIuJQjy6ZPn166D8C0adNK91m4cGHSWBdffHFSv5TxTj/99KSxZs+endRv1qxZpfukvmannVZ+HZZ6BGHqfIIp8ytedtllpfuMjAz+ibrX/GaZcvjNMuXwm2XK4TfLlMNvlimH3yxTDr9Zphx+s0w5/GaZcvjNMuXwm2XK4TfLVKsH9kydOjXpYJbLL7+8dJ/h4eHSfSDtwJ4ZM2a0NhbA5MmTS/dJOfgF0mucOnVq6T7Hjh1LGmvSpPJ/xqnTkKUcoAMwZ86c0n3mzp1buk+Z+rzmN8uUw2+WqZOGX9I9kt6S9FzXfXMkbZK0s7hM28Y2s1NmkDX/3wErRt23Hng8IhYDjxe3zWwCOWn4I+LfgLdH3b0auLe4fi9wS811mVnDUt/znx0RewGKy/n1lWRmbWj8oz5J64B1kPYRlZk1I3XN/6akBQDF5Vv9GnZP15XyeayZNSM1/I8Aa4vra4Fv1lOOmbVlkI/67gf+A7hE0h5JnwT+ELhR0k7gxuK2mU0gJ90Oj4hb+zy0vOZazKxF/oafWaYcfrNMtX5U36WXXlq63/XXX1+6T+r0VClHbR0+fDhprNQpo1J+t/nz076KMXPmzKR+KR/rTpkyJWmss88+u3Sf888/P2ms1Nds3759pfts3LixdJ+IGLit1/xmmXL4zTLl8JtlyuE3y5TDb5Yph98sUw6/WaYcfrNMOfxmmXL4zTLl8JtlyuE3y5TKHAhQ1dDQUKRMkzRr1qzSfRYtWlS6D5B04NFZZ52VNFbqlFEpU2+lToX10ksvJfUbGRkp3efgwYNJYy1btqx0n2uvvTZprEOHDiX1S/ndnnjiidJ9tm3bxsGDBzVIW6/5zTLl8JtlyuE3y1TqXH1fkrRD0jOSHpY0u9kyzaxuqXP1bQIuj4grgReAz9Vcl5k1LGmuvojYGBFHi5ubgYUN1GZmDarjPf9twGP9HpS0TtKIpJE2P1Y0s7FVOoGnpLuAo8B9/dpExAZgA3Q+568ynpnVJzn8ktYCK4Hl4VW62YSTFH5JK4DPAj8fET+ptyQza0PqXH1/DZwJbJK0TdJXGq7TzGqWOlffVxuoxcxa5G/4mWWq1aP6JHnHoFnDIsJH9ZlZfw6/WaYcfrNMOfxmmXL4zTLl8JtlyuE3y5TDb5Yph98sUw6/WaYcfrNMOfxmmXL4zTLl8JtlyuE3y5TDb5appOm6uh77tKSQNK+Z8sysKanTdSHpXOBG4JWaazKzFiRN11X4M+AzgE/NZTYBpZ63fxXwWkQ8LY19ujBJ64B1KeOYWXNKh1/SNOAu4KZB2ndP1+UTeJqNHyl7+y8CFgFPS3qZzgy9WyV9oM7CzKxZpdf8EfEsMP/47eIfwJKI+FGNdZlZw1Kn6zKzCc6Tdpj9jPGkHWY2JoffLFMOv1mmHH6zTDn8Zply+M0y5fCbZcrhN8uUw2+WKYffLFMOv1mmHH6zTDn8Zply+M0y5fCbZSrpBJ4V/Aj47z6PzSseP9Vcx4lcx4nGex3nD/oErZ7MYyySRiJiietwHa6jnTq82W+WKYffLFPjKfwbTnUBBddxItdxop+ZOsbNe34za9d4WvObWYtaDb+kFZJ+IGmXpPU9Hj9d0teLx7dIuqCBGs6V9F1J2yU9L+mOHm0+Imm/pG3Fz+/VXUfXWC9LerYYZ6TH45L0l8UyeUbSNTWPf0nX77lN0gFJd45q09jy6DUFvKQ5kjZJ2llcDvfpu7Zos1PS2gbq+JKkHcVyf1jS7D59x3wNa6jjC5Je61r+N/fpO2a+3iciWvkBhoAXgQuBKcDTwKWj2vwG8JXi+hrg6w3UsQC4prh+JvBCjzo+AnyrpeXyMjBvjMdvBh4DBFwHbGn4NXoDOL+t5QHcAFwDPNd13x8B64vr64Ev9ug3B9hdXA4X14drruMmYFJx/Yu96hjkNayhji8Anx7gtRszX6N/2lzzLwV2RcTuiDgCPACsHtVmNXBvcf1BYLlONg1wSRGxNyK2Ftd/DGwHzqlzjJqtBv4+OjYDsyUtaGis5cCLEdHvi1i1i95TwHf/HdwL3NKj6y8BmyLi7Yj4X2ATsKLOOiJiY0QcLW5upjMvZaP6LI9BDJKvE7QZ/nOAV7tu7+H9oftpm2Kh7wfmNlVQ8bbiamBLj4c/JOlpSY9JuqypGoAANkp6qpjOfLRBlltd1gD393msreUBcHZE7IXOP2u65obs0uZyAbiNzhZYLyd7Detwe/H2454+b4NKL482w99rDT76o4ZB2tRC0gzgG8CdEXFg1MNb6Wz6XgX8FfDPTdRQuD4irgE+CvympBtGl9qjT+3LRNIUYBXwTz0ebnN5DKrNv5W7gKPAfX2anOw1rOrLdGbH/jlgL/Anvcrscd+Yy6PN8O8Bzu26vRB4vV8bSZOAWaRtAo1J0mQ6wb8vIh4a/XhEHIiIg8X1R4HJkubVXUfx/K8Xl28BD9PZfOs2yHKrw0eBrRHxZo8aW1sehTePv7UpLt/q0aaV5VLsSFwJ/GoUb65HG+A1rCQi3oyIYxHxHvC3fZ6/9PJoM/xPAoslLSrWMmuAR0a1eQQ4vtf2Y8B3+i3wVMU+hK8C2yPiT/u0+cDxfQ2SltJZTv9TZx3Fc0+XdObx63R2MD03qtkjwK8Ve/2vA/Yf3ySu2a302eRva3l06f47WAt8s0ebbwM3SRouNoNvKu6rjaQVwGeBVRHxkz5tBnkNq9bRvY/nV/o8/yD5OlEdeyhL7Mm8mc7e9ReBu4r7fp/OwgU4g85m5y7gP4ELG6jhw3Q2h54BthU/NwOfAj5VtLkdeJ7OHtPNwLKGlseFxRhPF+MdXybdtQj4m2KZPQssaaCOaXTCPKvrvlaWB51/OHuBd+msvT5JZz/P48DO4nJO0XYJcHdX39uKv5VdwCcaqGMXnffRx/9Ojn8S9UHg0bFew5rr+IfitX+GTqAXjK6jX77G+vE3/Mwy5W/4mWXK4TfLlMNvlimH3yxTDr9Zphx+s0w5/GaZcvjNMvV/Rr089JEy5pQAAAAASUVORK5CYII=\n"
                    },
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2 id=\"Q2\">Questions 2</h2>\nCreate a Convolutional Neural Network class using ONE of the following constructors.  Train the network using the provided code then provide a screenshot of your training cost and accuracy with your validation data."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Constructor  using Batch Norm "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "class CNN_batch(nn.Module):\n    \n    # Contructor\n    def __init__(self, out_1=16, out_2=32,number_of_classes=10):\n        super(CNN_batch, self).__init__()\n        self.cnn1 = nn.Conv2d(in_channels=1, out_channels=out_1, kernel_size=5, padding=2)\n        self.conv1_bn = nn.BatchNorm2d(out_1)\n\n        self.maxpool1=nn.MaxPool2d(kernel_size=2)\n        \n        self.cnn2 = nn.Conv2d(in_channels=out_1, out_channels=out_2, kernel_size=5, stride=1, padding=2)\n        self.conv2_bn = nn.BatchNorm2d(out_2)\n\n        self.maxpool2=nn.MaxPool2d(kernel_size=2)\n        self.fc1 = nn.Linear(out_2 * 4 * 4, number_of_classes)\n        self.bn_fc1 = nn.BatchNorm1d(10)\n    \n    # Prediction\n    def forward(self, x):\n        x = self.cnn1(x)\n        x=self.conv1_bn(x)\n        x = torch.relu(x)\n        x = self.maxpool1(x)\n        x = self.cnn2(x)\n        x=self.conv2_bn(x)\n        x = torch.relu(x)\n        x = self.maxpool2(x)\n        x = x.view(x.size(0), -1)\n        x = self.fc1(x)\n        x=self.bn_fc1(x)\n        return x",
            "execution_count": 11,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Constructor  for regular Convolutional Neural Network"
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "class CNN(nn.Module):\n    \n    # Contructor\n    def __init__(self, out_1=16, out_2=32,number_of_classes=10):\n        super(CNN, self).__init__()\n        self.cnn1 = nn.Conv2d(in_channels=1, out_channels=out_1, kernel_size=5, padding=2)\n        self.maxpool1=nn.MaxPool2d(kernel_size=2)\n\n        self.cnn2 = nn.Conv2d(in_channels=out_1, out_channels=out_2, kernel_size=5, stride=1, padding=2)\n        self.maxpool2=nn.MaxPool2d(kernel_size=2)\n        self.fc1 = nn.Linear(out_2 * 4 * 4, number_of_classes)\n    \n    # Prediction\n    def forward(self, x):\n        x = self.cnn1(x)\n        x = torch.relu(x)\n        x = self.maxpool1(x)\n        x = self.cnn2(x)\n        x = torch.relu(x)\n        x = self.maxpool2(x)\n        x = x.view(x.size(0), -1)\n        x = self.fc1(x)\n        return x",
            "execution_count": 12,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "train loader  and validation loader "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "train_loader = torch.utils.data.DataLoader(dataset=dataset_train, batch_size=100 )\ntest_loader = torch.utils.data.DataLoader(dataset=dataset_val, batch_size=100 )",
            "execution_count": 13,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Convolutional Neural Network object "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "model = CNN(out_1=16, out_2=32,number_of_classes=10)\n#model =CNN_batch(out_1=16, out_2=32,number_of_classes=10)",
            "execution_count": 14,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Code used to train the model "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "import time\nstart_time = time.time()\n\ncost_list=[]\naccuracy_list=[]\nN_test=len(dataset_val)\nlearning_rate =0.1\noptimizer = torch.optim.SGD(model.parameters(), lr = learning_rate)\ncriterion = nn.CrossEntropyLoss()\nn_epochs=5\nfor epoch in range(n_epochs):\n    cost=0\n    model.train()\n    for x, y in train_loader:\n        optimizer.zero_grad()\n        z = model(x)\n        loss = criterion(z, y)\n        loss.backward()\n        optimizer.step()\n        cost+=loss.item()\n    correct=0\n    #perform a prediction on the validation  data \n    model.eval()\n    for x_test, y_test in test_loader:\n        z = model(x_test)\n        _, yhat = torch.max(z.data, 1)\n        correct += (yhat == y_test).sum().item()\n    accuracy = correct / N_test\n    accuracy_list.append(accuracy)\n    cost_list.append(cost)\n    \n",
            "execution_count": 15,
            "outputs": []
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "You will use the following to plot the Cost and accuracy for each epoch for the training and testing data, respectively. "
        },
        {
            "metadata": {},
            "cell_type": "code",
            "source": "fig, ax1 = plt.subplots()\ncolor = 'tab:red'\nax1.plot(cost_list, color=color)\nax1.set_xlabel('epoch', color=color)\nax1.set_ylabel('Cost', color=color)\nax1.tick_params(axis='y', color=color)\n    \nax2 = ax1.twinx()  \ncolor = 'tab:blue'\nax2.set_ylabel('accuracy', color=color) \nax2.set_xlabel('epoch', color=color)\nax2.plot( accuracy_list, color=color)\nax2.tick_params(axis='y', color=color)\nfig.tight_layout()",
            "execution_count": 16,
            "outputs": [
                {
                    "output_type": "display_data",
                    "data": {
                        "text/plain": "<Figure size 432x288 with 2 Axes>",
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3Xd4VFX6wPHvm15ID6GEEnqG3gQrrIQmUVBABQUFC03dta5x3d+u6xazDVd3RcUuiIiAgAaQYgEVC70lQIAAoYf0nsmc3x8zYIxIAmQyk+T9PM88zL333HvfAMmb98y554gxBqWUUsrdeLg6AKWUUup8NEEppZRyS5qglFJKuSVNUEoppdySJiillFJuSROUUkopt6QJSimllFvSBKWUUsotaYJSSinllrxcHcDl8PDwMP7+/q4OQyml3EphYaExxtT5AqROJyh/f38KCgpcHYZSSrkVESlydQw1oc5nWKWUUvWTJiillFJuSROUUkopt6QJSimllFvSBKWUUsotaYJSSinlljRBKaWUcksNMkGVHjlC/ldfuzoMpZQ6r8JSK5/uOkFGfomrQ3GpOv2g7qU68adnKU5Jof2a1Xj4+bk6HKWU4nhOEWuTT7Em+STf7D9DqdXG327pxh39W7k6NJdpkAkqctpUDk2YSPaCDwm/a6Krw1FKNUDGGHYezWV18knWJp9k17FcAFpHBDDxytbEWaK4IibcxVG6lhhjXB3DJQsMDDSXOtXRoYl3UXroEO1Wr8LD17eGI1NKqZ8rLivnm/0ZrN59is9STnIytwQPgT6tw4izNGGwJYp2jRshIpd1HxEpNMYE1lDYLtMgKyiAyAdmcHjSZHIWLyZs/HhXh6OUqqdO5RXzecopVu8+xdepGRSVlRPo48nATo2Ji23C9bFRhAf6uDpMt9RgKyhjDIfunEDZ8eO0/3Ql4qP/QZRSl88YQ8qJPNYmn2R18im2HckGIDrUn8GWKOIsTejfNhxfL0+nxaAVVB0nIkTOmMGR++4je8kSwm67zdUhKaXqqBJrOd8dyGRt8knWJJ/iaLZ9MvEeLUN5fGhH4ixNiG0adNlddw1Ng62gwP6bTtrt4yg/c4Z2K1cg3t41GJ1Sqj7LLCjl85RTrE05yZd7TlNQWo6ftwfXtm/MkM5RXB8bRVSQa0YJawVVD9irqOmkT5tOzrKPCR0z2tUhKaXclDGG/afzWZN8irXJJ9l0KAubgSbBvozsGc2QzlFc3S4SP2/ndd01NA26ggJHFTVmLOUF+bRLSkK8GnTOVkpVUFZu44e0TNY6klLamUIAujQPJs7ShCGWJnRpHoyHh3t13WkFVU+ICJEPzCD9gQfJXb6ckJEjXR2SUsqFcorK+GLPKdYmn+KLPafILbbi4+XB1e0iuPe6tsTFRtE81N/VYTYIDb6CAnsVdfDmWzClpbT95GPEU0t0pRqStIwC1iSfZG3yKX5Iy8RqM0QE+jAo1j7q7roOkQT61p3f57WCqkdEhMjp0zn68MPkrlxJSHy8q0NSSjlRuc2w5XCWYxaHU6SeygegU5MgpgxoS5ylCT1bhuLpZl13NUlEhgMvAJ7A68aYxErHWwHvAKGONgnGmOWOY92BV4FgwAZcYYwprvEYtYKyMzYbB0aORERos3Qp4tEg59FVqt7KL7Gybu9p1iSf5Is9p8ksKMXLQ7iybQRxligGW5rQMjzA1WHWiKoqKBHxBPYCQ4B04AdgvDFmd4U2s4EtxpiXRaQzsNwYEyMiXsBmYKIxZpuIRADZxpjymv46tIJyEA8PIqdP59hjj5O3eg3Bw4a6OiSl1GVKzyo8NwHrdwcyKS23EeLv7ei6i2JAx8YE+zXIx0v6AanGmAMAIjIfGAXsrtDGYK+QAEKAY473Q4HtxphtAMaYM84KUiuoCkx5OQduvAnx8aHNR4u1ilKqjrHZDNuP5rBm90nWJJ8k5UQeAG0jAxncuQlxsVH0aR2Gl2f9/t72DAgpbfnreTsq7Jqdlhg/++yGiIwFhhtj7nNsTwT6G2MerNCmGbAKCAMCgcHGmE0i8jDQB4gCGgPzjTH/cMbX4fQKylFKbgSOGmNuFJG3gYFAjqPJJGPMVrE/Yv0CMAIodOzf7Oz4fhKrpyeR06Zy7MkE8j//nKC4uNq8vVLqEhSVlvNVagZrdp/ksz2nOJ1nn4C1b0w4T4+wEGeJom3jRq4Os1bZinKtaYnxfS/Q5HwfrlWuVsYDbxtj/i0iVwFzRKQr9rxxLXAF9p/Va0VkkzFmbU3EXlFtdPH9Bkjmx1IR4AljzMJK7W4AOjhe/YGXHX/WquD4eE6/NIuMWS/TaNAgnZpEKTd0Mrf4XNfd16kZlFhtBPl6MaBTY4ZYmvCrTo0JDdD5NS8gHWhZYbsFP3bhnXUvMBzAGLNBRPyASMe5XxpjMgBEZDnQG6hbCUpEWgDxwF+BR6toPgp419j7HL8VkVARaWaMOe7MGCsTLy8ip07l+NNPU7BuHY0GDqzN2yulzsMYw65jueeS0o6j9g6YluH+jO/XiiGdm3BFTDg+XvW7664G/QB0EJE2wFFgHHBHpTaHgTjgbRGxAH7AaeBT4LciEgCUYu8Re94ZQTq7gvoP8FsgqNL+v4rIH7Bn3ARjTAkQDRyp0Cbdse8nCSo51jIFmAJgrFanBB0y8iYyZs3i9KxZBA4YoFWUUi5QXFbOhgNnWOsYCn48pxgR6NUylCeGdWJI5yZ0iLr8tZMaImOMVUQexJ5sPIE3jTG7RORZYKMxZhnwGPCaiDyCvftvkqOAyBKRmdiTnME+ui/JGXE6bZCEiNwIjDDGzBCRXwGPOz6DagacAHyA2cB+Y8yzIpIEPGeM+cpx/lrgt8aYTb90j5oeJFFR1gcLOPHHP9Ly9ddpdO01TrmHUuqnMvJL+CzFPq3Q+n0ZFJaWE+DjyXUdIomzNGFQbBSRjXSB0arog7pVuwYYKSIjsJeGwSIy1xgzwXG8RETeAh53bFenT7TWhNxyMxmvvELGrFkEXnO1/pamlBMYY9h7Mt8xi8NJthzJxhhoFuLH6N7RxFmacFXbCJ2AtYFyWoIyxjwFPAVQoYKacPZzJceovZuBnY5TlgEPOsbj9wdyavvzp4o8fHyIuP8+Tj77Zwq/+57AK2t9vIZS9VKp1cb3BzPtSSnlJEcy7WsndW8RwsNxHYmzRNGlebD+Uqhc8qDueyLSGPswx63ANMf+5diHmKdiH7o42QWx/UTomDGceeVVexWlCUqpS2azGb49eIZFm46yatcJ8kqs+Hp5cG37SKYPbE+cJYomwa5ZO0m5L31QtwqZ777Lyb89R+u5cwjoe6HHCpRSlR06U8CiTeks2nyUo9lFBPl6MbxrU4Z2acq17SPx99GuO2eoL59BaYKqgq2oiNQhQ/Hr2JFWb77h1HspVR/kFZexfMdxFm5K54e0LETg2vaRjO3TgqGdm2pSqgX1JUHpXHxV8PD3J+Keezj1j39QuGULAb16uTokpdxOuc3wzf4MFm1KZ+WuExSX2WjbOJDfDu/ELb2iaRai6yepi6cVVDXYCgtJjRuMX7eutJo9u+oTlGog9p/OZ9GmdD7acpTjOcUE+3lxU4/mjO3Tgp4tQ3Wgg4toBdWAeAQEED55MqdnzqRoxw78u3VzdUhKuUxOURmfbD/Gwk3pbDmcjYfAwI6NeTrewmBLEx0SrmqMVlDVVJ5fwP64OPz79KHlrJdq5Z5KuYtym2H9vtMs3JTOqt0nKbXa6NikEWN6t+CWXtFE6Qg8t6IVVAPj2SiQ8El3c/qFFynevRu/zp1dHZJSTrfvZB4LN6fz0eajnMorITTAm/FXtGRMnxZ0iw7RLjzlVFpBXYTyvDxSB8UReOWVtPjvi7V2X6VqU3ZhKcu2HWPRpnS2pefg6SFc36kxY/u04PrYKHy9tAvP3WkF1QB5BgURPnEiGbNmUbxnL36dOro6JKVqRFm5jXV77V14a5NPUVpuI7ZpEL+PtzCqZzSNg3T+O1X7tIK6SOXZ2aTGDabRwAFEz5xZq/dWqqYlH89l0aZ0lmw9SkZ+KeGBPtzcM5oxfaLp0jzE1eGpS6QVVAPlGRpK2IQJnJk9m8gHHsC3XTtXh6TURTmTX8KybfZReLuO5eLtKQyKjWJsn5b8qlNjvOv5cuiq7tAK6hJYs7JIjRtMUFwc0f/8R63fX6mLVWq18fmeUyzalM5nKaew2gxdo4MZ27sFI3tGEx6oq8/WJ1pBNWBeYWGEjR9H5ltv0/iBGfjExLg6JKV+5uwqtAs3pbNs2zEyC0qJbOTL5GtiGNOnBbFNg10dolIXpBXUJbJmZJA6eAjBN9xA8+f+5pIYlDqf03klLN16lIWb0kk5kYePpwdDOjdhTJ9oBnRojJd24dV79aWC0gR1GU4+l0jm3Lm0W7kCn5Ytqz5BKScpsZazNtnehffF3tOU2ww9WoYytk8LburejNAA7cJrSDRBuQFXJ6iyU6fYP3gIIaNG0uzPf3ZZHKphMsawPT3nXBdeTlEZTYJ9uaVXC8b2iaZ9VJCrQ1QuUl8SlH4GdRm8o6IIvfVWsj74gMhp0/COjnZ1SKoBOJlbzEdbjrJoUzr7TuXj6+XB0C5NGdunBde2j8TTQ2d3UPWDVlCXqezECfYPGUrI2DE0++MfXRqLqr+Ky8pZvfskCzels37faWwGercKZWyflsR3b0aIv7erQ1RuRCsoBYB306aEjBlNzsJFRE6dinfTpq4OSdUTxhg2H85m0eZ0Pt52jLxiK81C/Jj+q3aM6d2Cto0buTpEpZxKK6gaUHb0KKnDhhM2bhxNf/+0q8NRddyx7KJzXXgHMgrw8/bghq7NGNO7BVe1i9AuPFUlraDUOd7R0YTcPIrsBQuImHI/3lFRrg5J1TFFpeV8uusECzel8/X+DIyBfjHhTBvYjhu6NSXIT7vwVMOjFVQNKT18mP03jCB84kSaJDzp6nBUHWCMYeOhLBZuTCdpx3HyS6y0CPNndO8WjOkdTeuIOv8LsHKR6lRQIjIceAHwBF43xiRWOt4KeAcIdbRJMMYsF5EYIBnY42j6rTFmWs1+BXZaQdUQn1atCLnpJrLmzyfi/vvwiohwdUjKTaVnFbJ481EWbU7n0JlCAnw8GdHN3oXXv004HtqFp5xMRDyBl4AhQDrwg4gsM8bsrtDs98ACY8zLItIZWA7EOI7tN8b0dHacmqBqUMTUKeQsW0bmW28R9fjjrg5HuZGCEisrdp5g0aZ0Nhw4A8BVbSN4aFAHbujalEBf/VZUtaofkGqMOQAgIvOBUUDFBGWAs/NhhQDHajVCNEHVKN82bQgeMYLMee8Tfu+9eIWFuTok5UI2m+G7g5ks3JTOip3HKSwtp1V4AI8O6cgtvaJpGR7g6hBVPeXhH+wVk5C0scKu2WmJ8bMrbEcDRypspwP9K13mGWCViDwEBAKDKxxrIyJbgFzg98aY9TUWfAWaoGpY5LSp5CYlkfnOO0Q9/LCrw1EucOhMAYs2H2Xx5nTSs4po5OvFTd2bM7ZvC/q2DtNl0pXT2YpyrWmJ8X0v0OR8/wkrD0gYD7xtjPm3iFwFzBGRrsBxoJUx5oyI9AGWiEgXY0xuzUT/I01QNcy3fXuChg8ja85cIiZPxjNEF31rCPJLrCzffpyFm9L5Pi0TEbi2fSSPD+3EsC5N8ffRZdKVW0kHKk4g2oKfd+HdCwwHMMZsEBE/INIYcwoocezfJCL7gY7ARmqYJigniJw2nbwVK8l8dw6NH3rQ1eEoJzLGsGjzUf6StJvswjLaRgbyxLBO3NIrmuah/q4OT6lf8gPQQUTaAEeBccAdldocBuKAt0XEAvgBp0WkMZBpjCkXkbZAB+CAM4LUBOUEfp06EjRkMJnvvkv4pLvxDNJJO+ujw2cK+d1HO/gqNYM+rcP43YhYerfSLjzl/owxVhF5EPgU+xDyN40xu0TkWWCjMWYZ8Bjwmog8gr37b5IxxojIAOBZEbEC5cA0Y0ymM+LU56CcpHj3bg6OHkPjh39D5DSnPCKgXMRabuPNrw8yc/VevDw8eHJ4J+7s31qHhyu3UV9mktAE5URHps+gaPNm2q1di2ejOv9/RQE7j+aQsHg7O4/mMtgSxbOjumpXnnI79SVB6dKaThQ5YzrlOTlkvT/P1aGoy1RcVk7iihRGvfQ1J3JKeOmO3rx2V19NTko5kVZQTnb4/ikU79xJ+7Vr8AjQ517qom9SM3jqox0cOlPI7X1b8rsRFkICdG485b60glLVEjljOuVZWWR9sMDVoaiLlF1Yym8XbuOO178DYN59/fn72O6anJSqJU5PUCLiKSJbROQTx3YbEflORPaJyAci4uPY7+vYTnUcj3F2bLUhoFcvAq++ijNvvIGtuNjV4ahqMMbwyfZjDJ75JYs2H2XawHZ8+vAArm4f6erQlGpQaqOC+g32mW/P+jvwvDGmA5CF/WEwHH9mGWPaA8872tULkTNmUJ6RQfaCD10diqrC8Zwi7n93Iw/O20LTED+WPnANCTfE4uetD9oqVducmqBEpAUQD7zu2BZgELDQ0eQd4GbH+1GObRzH46SePFAS0LcvAVdcwZnXX8dWUuLqcNR52GyGORvSGDJzHV+lZvD0CAtLZlxD12idCUQpV3H2g7r/AX4LnH1SNQLINsZYHdvp2CcthAqTFzoeIstxtM+oeMHkWMsUYAqAsVqpKyIfmMHhSZPJWbyYsPHjXR2OqmDfyTwSFu9g06EsrusQyV9v7karCB3QopSrOS1BiciNwCnHXE2/Orv7PE1NNY6dY0lJng3MBpDAwDozBDGgf3/8e/cmY/ZrhI4Zg/j4uDqkBq/EWs7LX+znpc9TCfT14t+39mB072idCUIpN+HMLr5rgJEikgbMx9619x8gVETOJsaKExSem7zQcTwEcMr0Ga4gIkTOmIH1+HGylyxxdTgN3qZDmdz44lf8Z80+bujajDWPDmRMnxaanJRyI05LUMaYp4wxLYwxMdgnIvzMGHMn8Dkw1tHsbmCp4/0yxzaO45+ZuvyQ1nkEXnM1ft27c+bV2ZiyMleH0yDlFZfxh6U7GfvKBgpKrLw5qS8vju9FZCNfV4emlKrEFc9BPQk8KiKp2D9jesOx/w0gwrH/USDBBbE5lb2Kmk7Z0aPkLPvY1eE0OGuTTzL0+XXM+fYQd18Vw6pHBzIotomrw1JK/QKdSaKWGWNIGzOW8oJ82iUlIV46obyznc4r4ZmPd5G0/TgdmzQicUx3erfS1Y5V/aUzSahLIiJEPjCDskOHyV2+3NXh1GvGGBZsPMLgmV+yetdJHhvSkU8euk6Tk1J1hFZQLmCM4eDNt2BKS2n7yceIpz4EWtMOnSngqcU7+Gb/Ga6ICeO50d1pH9XI1WEpVSu0glKXTESInD6d0oMHyV250tXh1CvWchuvfLmfoc+vY3t6Dn+5uSsfTLlKk5NSdZBWUC5ibDYOjByJiNBm6VLEQ39XuFw7j+bw5KLt7DqWy5DOTfjzqK40DfFzdVhK1TqtoNRlEQ8PIqdPp2RfKnmr17g6nDqtqLSc55YnM+qlrzmVV8LLd/Zm9sQ+mpyUquO0gnIhU17OgRtvQnx8aPPRYq2iLsFX+zL43Uc7OJxZyPh+LUkYrms1KaUVlLps4ulJ5LSplOzZQ/7nn7s6nDolq6CUxz/cxoQ3vsPTQ3j//it5brSu1aRUfaIVlIsZq5X9I+LxDAoiZuGHOtVOFYwxfLz9OM9+vIuswjKmDmjLr+M66HIYSlWgFZSqEeLlReTUqRTv2kXBunWuDsetHc0u4t53NvLr97fQPNSfjx+8lt8O17WalKqvtIJyA6asjP3Db8AzMoKY+fO1iqqk3GaY++0h/rEyBZuBx4Z2ZNLVMXh56u9XSp2PVlCqxoi3NxFTplC8bTsFX3/j6nDcyt6TeYx95Rv+uGwXvVuHseqRAdx3XVtNTko1APpd7iZCbrkZr2bNyJg1i7pc1daUEms5M1fvJf7F9aRlFPD87T14955+tAzXhQSVqgkiMlxE9ohIqoj8bHJuEWklIp+LyBYR2S4iI85zPF9EHndWjJqg3ISHjw8R999H0ebNFH73vavDcamNaZmMeGE9L67dR3w3+1pNt/TStZqUqiki4gm8BNwAdAbGi0jnSs1+DywwxvTCvmTSrErHnwdWODNOTVBuJHTMGLwaNyZjVuX/Bw1DXnEZv1+yg7GvbKC4zMZbk6/gP+N6EaFrNSlV0/oBqcaYA8aYUuyLyo6q1MYAwY73Ify4uCwicjNwANjlzCB1rQc34uHrS8T993Hyb89RuHEjAX37ujqkWrN690n+b8lOTuYVM/maGB4f2olAX/3vqdSl8PAP9opJSNpYYdfstMT42RW2o4EjFbbTgf6VLvMMsEpEHgICgcEAIhKIfV2/IYDTuvdAE5TbCb31VjJmv0bGrJdp9eYbVZ9Qx53KK+aZZbtYvuMEsU2DeHlCb3rpchhKXRZbUa41LTH+Qr/hnq+/vPKH3+OBt40x/xaRq4A5ItIV+BPwvDEm39nd7pqg3IyHvz8R99zDqX/8g8ItWwjo1cvVITnF2bWa/pqUTLHVxhPDOjFlQFu8dXSeUrUhHWhZYbsFFbrwHO4FhgMYYzaIiB8Qib3SGisi/wBCAZuIFBtj/lfTQepzUG7IVlhIatxg/Lp1pdXs2VWfUMcczCjgd4t3sOHAGfq1Cee50d1o11iXw1CqplT1HJSIeAF7gTjgKPADcIcxZleFNiuAD4wxb4uIBVgLRJsKSUNEngHyjTH/csbXoRWUG/IICCB88mROz5xJ0Y4d+Hfr5uqQakRZuY3X1h/ghTX78PH04G+3dGPcFS3x8NDReUrVJmOMVUQeBD4FPIE3jTG7RORZYKMxZhnwGPCaiDyCvftvkqnlikYrKDdVnl/A/rg4/Pv0oeWsl1wdzmXbnp7Nk4t2kHw8l2FdmvDsqK40CdblMJRyhvoyk4RWUG7Ks1Eg4ZPu5vQLL1K8ezd+nSs/olA3FJZamblqL29+fZDIRr68MqE3w7s2c3VYSqlaEpOQtAh4E1iRlhhvu5hztYJyY+V5eaQOiiPwyitp8d8XXR3ORVu39zS/+2gH6VlFjO/XioQbYgnx1+UwlHI2d6qgYhKSBgOTgSuBD4G30xLjU6pzriYoN3f6xf+SMWsWbZYuxa9TR1eHUy1ZBaX8OWk3izcfpW1kIM+N7kb/thGuDkupBsOdEtRZMQlJIdiHrj+N/Rms14C5aYnxZb90jo7pdXPhd03EIzCQM6++4upQqmSMYenWowye+SXLth7jwevbs/w312lyUqqBi0lIigAmAfcBW4AXgN7A6gudp59BuTnP0FDCJkzgzOzZRD7wAL7t2rk6pPNKzyrk90t28sWe0/RoEcLc+/pjaRZc9YlKqXotJiFpMRALzAFuSkuMP+449EGl2S5+Rrv46gBrVhapcYMJiosj+p//cHU4P1FuM7zzTRr/WrUHY+DxYZ2YdHUMnjp0XCmXcacuvpiEpEFpifGfXcq52sVXB3iFhRE2fhy5SUmUpqW5OpxzUk7kMvrlb3j2k91cERPOqkcGcO+1bTQ5KaUqssQkJIWe3YhJSAqLSUiaUZ0TNUHVERGTJyM+PmS86vqZJYrLyvn3qj3c+OJXHMks5IVxPXl78hW6VpNS6nzuT0uMzz67kZYYnwXcX50TNUHVEV6RkYTdfjs5y5ZReuRI1Sc4yfcHMxnx4nr++1kqI3s0Z82jAxnVM1rXalJK/RKPmISkcz8gYhKSPAGfap3otJBUjQu/9x7E05MzLpifL7e4jN99tIPbXt1ASZmNd+7px8zbexIeWK3/Z0qphutTYEFMQlJcTELSIOB9YGV1TtRRfHWId1QUobfeStYHHxA5bRre0dG1ct+VO0/wx2U7OZ1Xwr3XtuHRIR11rSalVHU9CUwFpmNf5mMV8Hp1TtRRfHVM2YkT7B8ylJCxY2j2xz869V75JVYeX7CNlbvsazX9fUx3erQMrfpEpZRLudMovstRrV+Dk2MtcywpyROr2leRY+2QdYCv4z4LjTF/FJG3gYFAjqPpJGPMVrF/iPECMAIodOzffLFfUH3n3bQpIWNGk7NwEZFTp+LdtKnT7vXnj3ezavcJXatJKXXJYhKSOgDPAZ2BczNEpyXGt63q3Or+xOlScSM51uIJ9KninBJgkDGmB9ATGC4iVzqOPWGM6el4bXXsuwHo4HhNAV6uZmwNTuT992OM4czrzltxd/2+03yw8QhTBrTjgevba3JSSl2qt7D/PLcC1wPvYn9ot0oX/KmTHGt5KjnWkgd0T4615DpeecApYOmFzjV2+Y5Nb8frQv2Jo4B3Hed9C4SKiE57fR7e0dGE3DyK7AULKDt1qsavn19iJWHRDto2DuThwR1q/PpKqQbFPy0xfi0gaYnxh9IS458BBlXnxAt28VlSkp8DnkuOtTxnSUl+6mKjEhFPYBPQHnjJGPOdiEwH/ioif8C+QmOCMaYEiMY+geBZ6Y59xyteMznWMgV7hYWxWi82pHojcsoUcj5aQuabb9Ek4ckavfbfV6RwLKeIhdOuws/bs0avrZRqcIpjEpI8gH0xCUkPYl/BN6o6J1a33+aT5FhLIEByrGVCcqxlZnKspXVVJxljyo0xPbGvd99PRLoCT2Gfl+kKIBz7CA+wj+742SUq77CkJM+2pCT3taQk9xWvhjuSzKdVK0Juuoms+fOxnjlTY9fdsP8Mc749xOSr29CndXiNXVcp1WA9DAQAv8b+0dAE4O7qnFjdn/AvAz2SYy09gN8Cb2DvRxxYnZONMdki8gUwvMLa9SUi8hbwuGM7HWhZ4bQWwLFqxtcgRUydQs6yZWS+9RZRjz9e9QlVKCy18uSi7bSOCOCJYZ1qIEKlVEPmeCj3trTE+CeAfOzrQlVbdSsoqyUl2WD/nOgFS0ryC0DQhU4QkcYiEup47w8MBlLOfq7kGLV3M7DTccoy4C6xuxLIMcYcP8+llYNvmzYEjxhB5rz3sWZlXfb1/vXpXg5nFpI4ujv+Ptq1p5S6PGmJ8eVAn4ozSVyM6lZQecmxlqeAicB1jlF8VS2N2gx4x/E5lAewwBjziYh8JiKPre68AAAgAElEQVSNsXfpbQWmOdovxz7EPBX7MPOLyrQNVeS0qeQmJZH5zjtEPfzwJV9n06FM3vrmIBOvbM1V7XT9JqVUjdkCLI1JSPoQOPfgalpi/OKqTqxugroduAO4x5KSfCI51tIK+OeFTjDGbAd6nWf/eUdvGPsTww9UMx7l4Nu+PUHDh5E1Zy4RkyfjGRJy0dcoLivniYXbaR7iz5M3xDohSqVUAxYOnOGnI/cMUGWCqvZMEsmxlibYBzYAfG9JSa758c0XqSHOJHE+xXv2cnDUKCIfeIDGDz140ecnrkjhlS/3M+feflzXobETIlRK1ab6MpNEtRJUcqzlNuwV0xfYu+auA56wpCQvdGp0VdAE9aP0hx6i4NvvaP/ZWjyDLvjx4E9sO5LNLbO+5tY+Lfn72O5OjFApVVvcKUHFJCS9xXlGZKclxt9T1bnVHSTxNHCFJSX5bktK8l1AP+D/LipK5VSR06djy8sj6733qn1OibWcJxZuIyrIj6dvtDgxOqVUA/YJkOR4rQWCsY/oq1J1P4PyqNSldwZdqsOt+HXuTKPrryfzrbcJmzARz0ZV//L00mep7D2Zz5uT+hLsV9WYF6VUfSIiw7HPf+oJvG6MSax0vBXwDhDqaJNgjFkuIv2As2v+CPCMMeajX7pPWmL8oorbMQlJ7wNrqhNjdZPMyuRYy6fJsZZJybGWSdgz4fJqnqtqSeSM6ZTn5JD1/rwq2+46lsOsL/Yzulc0g2Kb1EJ0Sil34Rhd/RL2OVA7A+NFpHOlZr/HPvq6FzAOmOXYvxPo65iEYTjwqohczKwJHYBW1Wl4wYsmx1raA00sKclPJMdaRgPXYs+YG4Dq9yWpWuHfrRuB111H5ptvEX7nnXgEnH8J9rJyG098uJ3QAB/+cFPl/5NKqQagH5BqjDkAICLzsT/nurtCG4O9Ow4gBMfECcaYwgpt/LjwHKvEJCTlVWpzgh9nELqgqrLef4DfAVhSkhfjGBaYHGvp6zh2U3VuompP5IzpHBp/B1kfLCBi8qTztnn1y/3sPp7LKxP6EBqgK+IqVd94+Ad7xSQkbaywa3ZaYnzFpbjPN/dp/0qXeQZYJSIPAYHYJ1sAQET6A28CrYGJxphfnBg1LTG++qO2KqkqQcVYUpK3V95pSUnemBxribnUmyrnCejVi8Crr+LMG28QNn4cHn5+Pzm+92QeL65NJb57M4Z3dd5aUkop17EV5VrTEuP7XqBJdeY+HQ+8bYz5t4hcBcwRka7GGJsx5jugi4hYsE/IsMIYU3y+G8UkJN0CfJaWGJ/j2A4FfpWWGL+kqq+jqs+g/C5wzL+qiyvXiJw+nfKMDLIXfPiT/dZyG098uI1Gfl48O7LLL5ytlGoAqjP36b3AAgBjzAbs+SCyYgNjTDL22SG6XuBefzybnADSEuOzgWotB15VgvohOdZyf+WdybGWe7Evo6HcUMAVVxBwxRWcef11bCUl5/a/8dVBtqXn8MzILkQ08nVhhEopF/sB6CAibUTEB/sgiGWV2hwG4gAclZIfcNpxjpdjf2ugE5B2gXudL89Ua1BFVY0eBj5KjrXcyY8JqS/gA9xSnRso14h8YAaHJ00mZ/FiwsaPZ//pfP69ei9DOzfhpu66DqRSDZkxxioiDwKfYh9C/qYxZpeIPAtsNMYsAx4DXhORR7B3/00yxhgRuRZIEJEywAbMMMZkXOB2G2MSkmZiHzVogIeoZoFT3ZkkrufHEm6XJSX5s+pc3Nl0JolfZozh0J0TKDt+nLYrVnD7W5vYdyqf1Y8MICr4Qj23Sqm6zs1mkgjEPrHD2UEWq4C/piXGV/nDu9pz8bkjTVAXlr/+K47cfz+fPfhX/pnuy79v7cGYPi1cHZZSysncKUFdDp0Noh4LvPYaMntfzX8PefCrDpGM7h3t6pCUUg1MTELSasfIvbPbYTEJSZ9W51xNUPWYMfBCz7F42Kw8FXAU+xqRSilVqyIdI/cASEuMzwKiqnOiJqh67P0fDvN9po3pmZvwfPtVjPUXn6VTSilnscUkJJ2b2igmISmGKmafOOti5k9SdcjR7CKeW57CNe0juDtuEEcfWkLu8uWEjBzp6tCUUg3L08BXMQlJXzq2BwBTqnOiVlD1kDGGpxbvwGYMiaO7ExQXh2+nTmS8/AqmvNzV4SmlGpC0xPiV2B9P2gN8gH34elF1ztUEVQ99uCmddXtPk3BDLC3DAxAPDyKnT6f04EFyV650dXhKqQYkJiHpPuzrQD3meM3BPs9flTRB1TMnc4v58ye76dcmnAn9W5/bHzR0CD7t23HmlVcwNpsLI1RKNTC/Aa4ADqUlxl8P9AJOV+dETVD1iDGGpz/aQVm5jX+M6Y6Hx4+j9sTDg8hp0ynZl0re6mqtFaaUUjWhOC0xvhggJiHJNy0xPgX79EhV0gRVjyzbdow1yad4fGgnYiJ//oxe8A3D8YmJIWPWLK2ilFK1Jd3xHNQSYHVMQtJSfj4x7XnpTBL1xOm8EoY8/yVtIgNZOO1qPD3O/8xTztKlHHsygRYv/Y+guLhajlIpVRvcdSaJmISkgdgXP1yZlhhfWlV7TVD1xIz3NrEm+RTLf30t7aN+eX0wY7Wyf0Q8nkFBxCz8UB/eVaoectcEdbG0i68eWL7jOMt3nODhwR0umJwAxMuLyKlTKN61i4J162opQqWUuniaoOq4zIJS/rB0J92iQ5hyXdtqnRMyciTezZtzetYs6nIFrZSq3zRB1XF/+ngXOUVl/PPW7nh5Vu+fU7y9iZg6leJt2yn4+hsnR6iUUpdGE1Qdtnr3SZZuPcYD17cntmnwRZ0bcsvNeDVrZh/Rp1WUUsoNaYKqo3IKy3j6ox3ENg1ixq/aX/T5Hj4+RNx3L0WbN1P43fdOiFAppS6PJqg66i9JuzlTUMq/bu2Bj9el/TOGjh2LV+PGZMyaVcPRKaXU5dMEVQd9secUH25KZ9rAtnSNDrnk63j4+hJx/30Ufv89hRs31mCESil1+TRB1TF5xWX8bvEO2kc14qFBHS77eqG33opnZCQZs16ugeiUUqrmOC1BiYifiHwvIttEZJeI/Mmxv42IfCci+0TkAxHxcez3dWynOo7HOCu2uuy5FSmcyC3mn2O74+ftednX8/D3J2LyZAq++YbCLVtqIEKllKoZzqygSoBBxpgeQE9guIhcCfwdeN4Y0wHIAu51tL8XyDLGtAeed7RTFXyTmsG87w5z77Vt6NUqrMauGzbudjzDwsh4WasopZT7cFqCMnb5jk1vx8sAg4CFjv3vADc73o9ybOM4Hic6D885BSVWnly8nTaRgTw2tFoTAVebR2Ag4ZMnU7BuPUU7dtTotZVS6lI59TMoEfEUka3AKWA1sB/INsZYHU3SgWjH+2jgCIDjeA4QUfmaybGWKcmxlo3JsZaNxmqtfLje+uene0jPKuLvY2qma6+ysDvuwDMkhIyXX6nxayul1KVwaoIyxpQbY3oCLYB+gOV8zRx/nq9a+tkTpJaU5NmWlOS+lpTkvuLlVXPBurEf0jJ5Z0Mad18VQ7824U65h2ejQMLuvov8zz6jePdup9xDKeU+RGS4iOxxfO6fcJ7jrUTkcxHZIiLbRWSEY/8QEdkkIjscfw5yVoy1MorPGJMNfAFcCYSKyNnM0oIf1wVJB1oCOI6HAJm1EZ87Ky4r57cLt9MizJ8nhtVs115l4RMm4BEUpFWUUvWciHgCLwE3AJ2B8SLSuVKz3wMLjDG9gHHA2QcmM4CbjDHdgLuxL+HuFM4cxddYREId7/2BwUAy8Dkw1tHsbmCp4/0yxzaO458ZnYOHmav3cjCjgMTR3Qn0dW7F6BkcTPjEieStXk3xnr1OvZdSyqX6AanGmAPGmFJgPvZxABUZ4OwcaiE4igljzBZjzNnCYhfgJyK+zgjSmT/xmgHvODK1B/ZM/ImI7Abmi8hfgC3AG472bwBzRCQVe+U0zomx1QlbDmfx+voDjO/XimvaR9bKPcPvmkjmO+9w5tVXiJ45s1buqZSqWR7+wV4xCUkVn76fnZYYP7vC9rnP/B3Sgf6VLvMMsEpEHgICsRcZlY0BthhjSi4/6p/TBQvdVIm1nBtf/Ir8EiurHhlAkJ93rd371MznOfPaa7T95GN827WrtfsqpWpGVQsWisitwDBjzH2O7YlAP2PMQxXaPIo9R/xbRK7CXkR0NcbYHMe7YO/5GmqM2e+Mr0NnknBT/12byr5T+Tw3ulutJieA8MmTEH9/Ml55tVbvq5SqNec+83eoOB7grHuBBQDGmA2AHxAJICItgI+Au5yVnEATlFvaeTSHl7/cz9g+LfhVp6hav79XWBhh48eRm5REaVpard9fKeV0PwAdHDP7+GD/SGVZpTaHgTgAEbFgT1CnHWMLkoCnjDFfOzNITVBuptRq4/EPtxER6MP/xVceVFN7IiZPRnx8yHh1dtWNlVJ1iuNZ0weBT7EPXltgjNklIs+KyEhHs8eA+0VkG/A+MMkxcO1BoD3wfyKy1fFyym/S+hmUm3lhzT6eX7OX1+7qy5DOTVway8nnniNz7ntETp1C6O3j8G5S+9WcUuriVfUZVF2hFZQbSTmRy/8+38fIHs1dnpwAIqZNo9HAgWS8/AqpcXEcffwJirZtc3VYSqkGQisoN2Ett3HLrG84ll3E6kcHEh7o4+qQzik9dIisefPIXrQYW34+ft27Ez5xIsHDhiI+7hOnUsquvlRQmqDcxMtf7OfvK1N46Y7exHdv5upwzqs8v4CcJUvImjOH0kOH8GrcmNDx4wi77Ta8ImvnOS2lVNU0QbmB+pKgUk/lM+LF9cTFRvHyhD6uDqdKxmaj4KuvyJwzl4L16xFvb4Lj4wmbOAH/Ll1cHZ5SDZ4mKDdQHxJUuc1w6yvfcCCjgNWPDKRxkFNmDHGakgMHyJr7HtlLlmAKC/Hv04fwiRMIGjyYhjKZr1LuRhOUG6gPCer19Qf4S1Iy/7m9Jzf3iq76BDdVnptL9uLFZM19j7L0dLyaNiXsjjsIvXUsXmE1t7iiUqpqmqDcQF1PUGkZBQx/YR3XtIvk9bv7Uh/WZzTl5eR/+SWZc+ZQuOFbxNeXkJE3ETZhAn6dnDsbu1LKThOUG6jLCcpmM4x77VuSj+ey+pGBNA3xc3VINa54716y5r5HzrJlmOJiAvr1I/yuiTS6/nrEs+YXXVRK2WmCcgN1OUHN2ZDG/y3dxT/GdOe2K1pW2b4uK8/OJnvhQjLfm4f1+HG8o6MJu/NOQseMxjMkxNXhKVXvaIJyA3U1QR3JLGTYf9bRp3UY797Tr1507VWHsVrJW/sZWXPmULhxI+LvT8iokYRPnKizpitVgzRBuYG6mKCMMUx843u2HM5i1aMDiQ71d3VILlG8ezeZc98j95NPMKWlBF59NWF3TaTRgAGIh05wotTl0ATlBupigpr//WESFu/gLzd3ZcKVrV0djstZMzPJXrCArHnvYz11Cu/WrQi/cwIho2/Bs1EjV4enVJ2kCcoN1LUEdTyniKEz19ElOph5912Jh0fD6NqrDlNWRu6qVWTNmUvR1q14BAYSMno04XfegU9MjKvDU6pO0QTlBupSgjLGcM/bP/DtgUxWPnwdrSPq/P8dpynasYPMOXPIXbESysoIHDiA8Il3EXjN1Q3m8zqlLocmKDdQlxLU4s3pPLpgG3+4sTP3XNvG1eHUCdbTp8ma/wFZ8+dTfuYMPm3bEj5xAiEjR+IRWOe/95RyGk1QbqCuJKhTucUMeX4dHaIasWDqVdq1d5FspaXkrVhB5rtzKN61C4+gIELHjCFswp34tGjh6vCUcjuaoNxAXUhQxhimztnEl3tPs/w319GusX7wf6mMMRRt2UrW3DnkfroKbDYaDRpE+MSJBPRvOMP1laqKJig3UBcS1MfbjvHQ+1t46oZYpg7UZ31qStmJE2S9P5/sDz6gPDsb344dCZs4gZAbb8TDv2EO3VfqLE1QbsDdE9SZ/BKGPL+OlmH+LJp+NV6e+nxPTbMVF5OblETmu3Mo2bMHz5AQQm+7lbA77sC7mXuuq6WUs2mCcgPunqAenLeZT3edIOnX19GxSZCrw6nXjDEU/vADWXPmkrd2LYgQNHgw4XdNxL93b+3+Uw2KJig34M4JauXOE0ybu4nHhnTkobgOrg6nQSk7epTMefPI/nAhttxcfDtbCJ94F8EjbsDDt26tt6XUpdAE5QbcNUFlF5YyeOY6ooJ8WfrgNXhr155L2AoLyVn2MZlz51Cauh/P8HBCb7+NsHHj8W4S5erwlHIaTVBuwF0T1KMLtrJs6zGWPngNXZrrbN2uZoyhcMMGMufMJf+LL8DTk+Bhw+zdfz16uDo8pWqcJig34I4J6rOUk9zz9kZ+Pag9jw7VBfrcTenhw2S99x7ZixZjy8/Hr3t3widOIHjYMMTHx9XhKVUjNEG5AXdLULnFZQyduY5gfy8+fuhafL10UT53VZ5fQM6SJWTNnUtpWhqejSMJGzeOsNtvxysy0tXhKXVZqpOgRGQ48ALgCbxujEmsdLwV8A4Q6miTYIxZLiIRwELgCuBtY8yDzvgaQBNUjUpYtJ0FG4/w0Yxr6NEy1NXhqGowNhsFX31F5py5FKxfj3h7EzxiBGF3TcS/SxdXh6fUJakqQYmIJ7AXGAKkAz8A440xuyu0mQ1sMca8LCKdgeXGmBgRCQR6AV2Brs5MUF7OunBDs37faeb/cISpA9tqcqpDxMODRgMG0GjAAEoOHCRr7lyylywhZ+lS/Hv3JnziBIIGD0a8vV0dqlI1qR+Qaow5ACAi84FRwO4KbQwQ7HgfAhwDMMYUAF+JSHtnB6kVVA3IL7Ey7Pl1+Hp7sPzX1+HnrV17dVl5Xh7ZixaR9d48yo4cwatpU8LGjyf0tlvxCgtzdXhKVckzIKS05a/n7aiwa3ZaYvzssxsiMhYYboy5z7E9EehfsRoSkWbAKiAMCAQGG2M2VTg+CeirXXy/wF0S1B+W7mTOt4dYOO0q+rQOd3U4qoaY8nLyv1xH5px3KdzwLeLrS/BNNxI+cSJ+nXQAjHJf1ejiuxUYVilB9TPGPFShzaPYc8S/ReQq4A3sXXo2x/FJODlBOe0BHRFpKSKfi0iyiOwSkd849j8jIkdFZKvjNaLCOU+JSKqI7BGRYc6KrSZ9e+AM7244xOSr22hyqmfE05OgQdfT+q23aLNsKSGjRpH7SRIHR93MobvuJnf1akx5uavDVOpSpAMtK2y3wNGFV8G9wAIAY8wGwA+o1RFETqugHOVhM2PMZhEJAjYBNwO3AfnGmH9Vat8ZeB9732hzYA3Q0Rjziz8BXF1BFZWWM/yFdRgDKx++jgAf/UivvivPziZ74UIy583Deuw43s2bE3bnnYSOHYNniD7zptxDNSooL+yDJOKAo9gHSdxhjNlVoc0K4ANjzNsiYgHWAtHGkTTqVRefiCwF/gdcw/kT1FMAxpjnHNufAs84Mvd5uTpB/fmT3bzx1UHev/9KrmoX4bI4VO0zVit5n31G1rtzKNy4EfH3J/Caqwno2RP/nj3x69oVDz8/V4epGqhqDjMfAfwH+xDyN40xfxWRZ4GNxphljqLhNaAR9gETvzXGrHKcm4Z9AIUPkA0MrTgCsMa+jtpIUCISA6zDPizxUWASkAtsBB4zxmSJyP+Ab40xcx3nvAGsMMYsrHit5FjLFGAKQJ+DB/oUlpQ4Pf7z2XQoi7GvfMOd/Vvxl5u7uSQG5R6Kk5PJmvc+Bd99R9nhw/adXl74xcbi70hY/j174h3dXCetVbVCH9St7g1EGgFfAn81xiwWkSZABvaM/Gfs3YD3iMhLwIZKCWq5MWbRL13bVRVUcVk58S+up7jMxqePDKCRr3btKTvrmTMUbdtG0ZatFG3dStHOnZiiIgA8G0eeq7D8e/bEr0sXrbKUU9SXBOXUn6wi4g0sAt4zxiwGMMacrHD8NeATx2Z1PrRzCy+s3cf+0wW8e08/TU7qJ7wiIggaNIigQYMAe1dgyd69FG51JKyt28hbvcbRWKsspS7EmYMkBPs0GZnGmIcr7G9mjDnueP8I9rH340SkCzCPHwdJrAU6uNsgie3p2dwy6xvG9m7B38d2r9V7q/pBqyzlbPWlgnJmgroWWA/sAGyO3b8DxgM9sXfxpQFTKySsp4F7ACvwsDFmxYXuUdsJqtRq46b/fkV2USmrHhlIiL/OLqAun7FaKd6zx560HFXWTz7Lsljw79FDqyxVbZqg3EBtJ6iZq/fy4tp9vHF3X+IsTWrtvqrh0SpLXQ5NUG6gNhPU7mO5jPzfV9zUoznP396zVu6p1FnnqixHhVW0Tass9cs0QbmB2kpQZeU2bn7pa07mlrDm0QGEBui6Qcr1tMpSv0QTlBuorQT10uep/PPTPbwyoTfDuzZz+v2UuhQ/q7K2bqXsyBH7wbNVVs+e+PfsgX8PrbLqM01QbqA2EtS+k3nEv/gVQ7o04aU7ejv1XkrVNK2yGiZNUG7A2Qmq3GYY8/I3HM4sZNUjA4hs5Ou0eylVGy6mygro2ROv5lpl1UWaoNyAsxPUa+sO8Nflybw4vhcjezR32n2UcqULVVlejRvbuwS1yqpTNEG5AWcmqAOn87nhhfUM6NiY2RP76G+RqsG4YJXl7V1h9gutstyVJig34KwEZbMZbp+9gT0n8ljz6ECigvU3RtWwWTMyfvIgsVZZ7k0TlBtwVoJ6++uDPPPxbv51aw/G9mlR49dXqq4zZWUU792rVZab0gTlBpyRoI5kFjL0+XX0bxvOW5Ou0G8qparpJ1XWFsdnWcXFwNkqy56w/Lp2w6dNDF6NG+v3l5NognIDNZ2gjDHc+fp3bE/PYdUjA2ge6l9j11aqoblglQV4BATg3bo1PhVfMfY/PcPDNXldBk1QbqCmE9S87w7zu4928Nzobozv16rGrquUsrNmZFCcnELpoUOOVxqlhw5Rln4Uyn9cuMAjKOi8icundWs8Q0Nd+BXUDZqg3EBNJqij2UUMe34dPVqGMPfe/vrbm1K1yJSVUXb06I+JKy2N0jT7+7Jjx6DCzynPkBC8Y1rjGxNToQKLwSemNZ6NGrnwq3AfmqDcQE0lKGMMk976gR/SMvn04QG0DA+ogeiUUjXBVlpK2ZEjjsR16CdJzHrixE/aekZEVKq87InLp1UrPAIazve1Jig3UFMJ6sONR3hi4Xb+NLILd18dc/mBKaVqha2oiNLDR851FZYeOkSZI4lZT5/+SVuvqKifdBeeq75atap3w+I1QbmBmkhQJ3OLGTLzS2KbBjN/ypV4eGjXnlL1QXl+AWWHK1ZcP74vz8z8saEIXs2aVqi8Yn5MZC1aID51b/UCTVBu4HITlDGG+9/dxPp9p1n58ADaRNb5f0+lVDWU5+ZSeujwjwM1KiQvW07Ojw09PPBu3tzeVVhpwIZ3dDTi5eW6L+IC6kuCcs+/3VqybNsx1iSf5PfxFk1OSjUgnsHB+Hfrin+3rj87Zs3KouzQzyuvnC1bsFX8hdjLC5/o6PMO2PBu1hTx9KzFr6h+arAV1Om8EoY+/yUxkYEsnHY1ntq1p5S6AGMM5WfO/HywhuN1duonAPH2xrtVq/MO2PCKikI8PJwaq1ZQddwzy3ZRUFLOP8d21+SklKqSiOAVGYlXZCQBffr85JgxBuup0/bh8RUHbBw6RMFXX2FKS3+8jp8fPmeTV4Xnu7xbt67V2TVEZDjwAuAJvG6MSax0vBXwDhDqaJNgjFnuOPYUcC9QDvzaGPOpU2JsiBXUih3Hmf7eZp4Y1okHrm/vhMiUUsrO2GxYT5z42fNdpYcOUZqeDmVl59p6BATgXSFpBd8wAr9OHS/6nlVVUCLiCewFhgDpwA/AeGPM7gptZgNbjDEvi0hnYLkxJsbx/n2gH9AcWAN0NMaUV77P5WqQFdSSrUfpFh3C1AFtXR2KUqqeE8dAC+/mzQm86qqfHDNWK2XHj//s+a7iXbvJW7UaP0vnS0pQ1dAPSDXGHAAQkfnAKGB3hTYGCHa8DwGOOd6PAuYbY0qAgyKS6rjehpoOskEmqFl39iGzoBQvT+f2Ayul1IWIlxc+LVvi07IlXHftT46ZsrKfzKBxMTz8g71iEpI2Vtg1Oy0xfnaF7WjgSIXtdKB/pcs8A6wSkYeAQGBwhXO/rXRu9CUFWoUGmaA8PYTGQbp8u1LKfYm39yWfayvKtaYlxve90OXPs69yNhwPvG2M+beIXAXMEZGu1Ty3RjTIBKWUUg1cOtCywnYLfuzCO+teYDiAMWaDiPgBkdU8t0ZoH5dSSjU8PwAdRKSNiPgA44BlldocBuIARMQC+AGnHe3GiYiviLQBOgDfOyNIraCUUqqBMcZYReRB4FPsQ8jfNMbsEpFngY3GmGXAY8BrIvII9i68ScY+7HuXiCzAPqDCCjzgjBF80ECHmSulVH1WXx7U1S4+pZRSbkkTlFJKKbekCUoppZRb0gSllFLKLdXpQRIiYgOKqmx4fl7YR6C4E42p+twxLo2pejSm6rvUuPyNMXW+AKnTCepyiMhGY8yFnrSudRpT9bljXBpT9WhM1eeucdWWOp9hlVJK1U+aoJRSSrmlhpygZlfdpNZpTNXnjnFpTNWjMVWfu8ZVKxrsZ1BKKaXcW0OuoJRSSrkxTVBKKaXcUr1PUCIyXET2iEiqiCSc57iviHzgOP6diMS4QUyTROS0iGx1vO6rhZjeFJFTIrLzF46LiLzoiHm7iPR2g5h+JSI5Ff6e/lALMbUUkc9FJFlEdonIb87Tplb/rqoZU63+XYmIn4h8LyLbHDH96TxtavV7r5ox1fr3nuO+niKyRUQ+Oc+xWv8Z5TaMMfX2hX0a+f1AW8AH2AZ0rtRmBvCK4/044AM3iGkS8L9a/rsaAPQGdv7C8RHACuyraV4JfOcGMf0K+KSW/56aAf/f3v2/SlGFcRx/P+UtNEUpjTvD+9MAAAUwSURBVERLg4I5KX4LxLoElf0QFBqkKKVJ/RaFSD8URlH4B0QQgdIXuKZ9NYtbGH2TDH/ITBGkzhAiQZcES+2aRZb69MOMta57r0u0Z8bt84ILs7Nnd5557n32zJ6Ze2ZOuTwG+LbF7y9prtqMKWmuyn0fXS73ADuAeU1tUtdeOzElr71yu48Ar7b6HaXOU51+uv0b1Fxgn7vvd/c/gNeBhU1tFgJ95fImYL6ZtbqlccqYknP3z4HDwzRZCKz3whfAODObWHFMybn7AXffXS7/AkRgUlOzpLlqM6akyn0/Vj7sKX+ar8hKWnttxpScmU0G7gBeHKJJ6s+o2uj2DmoS8H3D4wHOLty/27j7CWAQuKzimADuLoeHNpnZlS2eT63duFO7oRyy+cDMpqXccDnUMpviSLxRZbkaJiZInKty2GoPcBD42N2HzFOi2msnJkhfe88CjwKnhng+eZ7qots7qFZHGc1HTO20+S+1s733gKnuPgP4hH+OnqqUOk/t2A1McfeZwHPAu6k2bGajgbeBVe5+tPnpFi/peK7OEVPyXLn7SXefBUwG5prZ9OaQW72s4piS1p6Z3QkcdPddwzVrsa7q2kui2zuoAaDxCGgy8MNQbcxsBDCWzg4rnTMmdz/k7sfLhy8A13cwnna1k8uk3P3o6SEbd98C9JjZ+E5v18x6KDqCje6+uUWT5Lk6V0xV5arc3s/AZ8DtTU+lrr1zxlRB7fUCC8zsO4rh/lvNbENTm8ryVLVu76B2Atea2dVmdhHFCcb+pjb9wIpyeRGw1cuzkVXF1HS+YgHFOYWq9QP3lVeozQMG3f1AlQGZ2RWnx+LNbC7F3/OhDm/TgJeA6O7PDNEsaa7aiSl1rsxsgpmNK5dHArcBeVOzpLXXTkypa8/dV7v7ZHefSvFZsNXdlzU1S/0ZVRsjqg6gk9z9hJk9DHxIcfXcy+7+tZmtAb5y936Kwn7FzPZRHJUsrUFMK81sAcU0+4cprizqKDN7jeJKr/FmNgA8RXESGXdfC2yhuDptH/AbcH8NYloEPGhmJyhuu7I0QeH2AsuBveW5DIDHgasa4kqdq3ZiSp2riUCfmV1I0Rm+6e7vV1l7bcaUvPZaqThPtaGpjkREpJa6fYhPRETOU+qgRESkltRBiYhILamDEhGRWlIHJSIitaQOSiSRmIWbYxbOmq1aRFpTByUiIrWk/4MSaRKzsAxYSXE7lB0UtzsYBNYBtwBHgKUhjz/GLMwC1gKjKG6j8kDI45GYhWvK9ROAk8BiiulqngZ+AqYDu4BlIY8qQpEW9A1KpEHMQgCWAL0hj7MoOpd7gUuA3SGPc4BtFLNaAKwHHgt5nAHsbVi/EXg+5HEmcCNweqqj2cAq4DqKe4L1dnynRM5TXT3Vkci/MJ9igtCdMQsAIyluzXAKeKNsswHYHLMwFhgX8ritXN8HvBWzMAaYFPL4DkDI4+8A5ft9GfI4UD7eA0wFtnd+t0TOP+qgRM5kQF/I4+rGlTELTza1G25YbribyR1vWD6JalBkSBriEznTp8CimIXLAWIWLo1ZmEJRK4vKNvcA20MeB4EjMQs3leuXA9tCHo8CAzELd5XvcXHMwqikeyHSBXT0JtIg5PGbmIUngI9iFi4A/gQeAn4FpsUs7KK4YGJJ+ZIVwNqyA9rPPzOXLwfWxSysKd9jccLdEOkKuopPpA0xC8dCHkdXHYfI/4mG+EREpJb0DUpERGpJ36BERKSW1EGJiEgtqYMSEZFaUgclIiK1pA5KRERq6S+JhPzc0GjQPAAAAABJRU5ErkJggg==\n"
                    },
                    "metadata": {
                        "needs_background": "light"
                    }
                }
            ]
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "dataset: https://github.com/zalandoresearch/fashion-mnist"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<h2>About the Authors:</h2> \n\n<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD."
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Other contributors: <a href=\"https://www.linkedin.com/in/michelleccarey/\">Michelle Carey</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a> "
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "<hr>"
        },
        {
            "metadata": {},
            "cell_type": "markdown",
            "source": "Copyright &copy; 2018 <a href=\"cognitiveclass.ai?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu\">cognitiveclass.ai</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>."
        }
    ],
    "metadata": {
        "kernelspec": {
            "name": "python3",
            "display_name": "Python 3.6",
            "language": "python"
        },
        "language_info": {
            "name": "python",
            "version": "3.6.9",
            "mimetype": "text/x-python",
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "pygments_lexer": "ipython3",
            "nbconvert_exporter": "python",
            "file_extension": ".py"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}